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CHAPTER  1: 

Introduction 


Demand  exists  for  the  mandatory  enforcement  of  confidentiality  and  integrity  policies,  espe¬ 
cially  in  the  military  and  business  domains.  The  need  to  share  information  across  different 
sensitivity  domains  necessitates  the  development  of  multilevel  distributed  security  architectures 
that  enforce  information  flow  policies,  while  maintaining  usability  and  efficiency.  Military  and 
commercial  systems  are  currently  unable  to  provide  high  assurance  support  for  multilevel  secu¬ 
rity  policy  enforcement  [1].  With  adversaries  and  hackers  performing  increasingly  sophisticated 
attacks,  the  lack  of  robust  security  measures  are  a  cause  for  concern  due  to  the  risk  of  leakage 
of  sensitive  information,  corruption  of  critical  data  and  denial  of  service.  New  trusted  com¬ 
puting  approaches  involving  interoperable  system  security  features  and  standardized  security 
mechanisms  are  required  to  secure  mission-critical  systems. 

MYSEA  provides  simultaneous  access  to  different  security  classifications  of  information  using 
standardized  commercial-off-the-shelf  components.  Mandatory  enforcement  of  confidentiality 
and  integrity  policies  necessitates  the  design  of  distributed  architectures  to  control  information 
flow  between  systems  while  providing  for  security  controls  to  protect  against  malicious  code 
and  attacks  from  adversaries. 

1.1  Motivation 

The  confidentiality,  integrity  and  availability  (CIA)  of  data,  applications  and  networks  are  vital 
to  any  organization.  Mandatory  access  controls,  where  data  is  segregated  into  security  classifi¬ 
cations  and  is  only  accessible  to  suitably  authorized  personnel  are  a  way  of  addressing  security 
concerns  associated  with  malicious  software.  The  need  to  protect  sensitive  data  and  allow  se¬ 
lective  sharing  of  information  between  users  necessitates  the  development  of  high  assurance 
trusted  systems.  For  users  to  embrace  the  security  controls  imposed,  systems  need  to  be  ef¬ 
ficient  and  user-friendly.  Therefore,  there  is  a  need  to  investigate  the  system  performance  of 
MYSEA,  to  identify  and  analyze  the  potential  bottlenecks. 

1.2  Purpose  of  Study 

Various  trusted  security  components  are  part  of  MYSEA.  These  amend  the  underlying  platform 
to  provide  authorized  users  with  simultaneous  access  to  information  of  different  classification 
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levels,  introducing  necessary  overhead  to  the  system.  There  is  a  need  to  investigate  the  “cost  of 
security”  and  study  the  performance  overhead  associated  with  high  assurance  systems  following 
this  design,  compared  with  systems  operating  traditional  servers.  This  is  especially  crucial  for 
multi-user  distributed  systems  where  scaling  and  user-friendliness  become  important  factors  in 
user-acceptance  and  ergonomic  security,  both  of  which  are  stated  MYSEA  goals. 

To  create  a  trusted  security  environment,  the  MYSEA  design  introduces  some  (perhaps  sig¬ 
nificant)  overhead  that  may  impact  performance.  Every  TCP  connection  (for  User  Datagram 
Protocol  (UDP),  every  packet)  from  the  multilevel  Local  Area  Network  (LAN)  is  processed, 
verified  and  cross-referenced  against  access  control  permissions  by  a  trusted  subject.  This  in¬ 
spection  takes  place  in  different  processes  in  the  server  architecture.  Key  Performance  Indica¬ 
tor  (KPI)s  associated  with  the  system  need  to  be  measured  and  their  impacts  assessed.  The  “cost 
of  security”  in  terms  of  efficiency,  performance  and  responsiveness  is  an  important  factor  in  the 
application  and  risk  analysis  of  security  controls.  Thus,  measurement  of  the  KPIs  is  needed 
before  an  informed  tradeoff  between  the  performance  penalties  associated  with  implementing 
MYSEA  and  the  security  requirements  and  policies  mandated. 

The  goal  of  our  study  is  to  analyze  the  performance  of  selected  aspects  of  MYSEA  and,  when 
applicable,  identify  system  performance  bottlenecks.  System  usability  is  determined  by  many 
factors,  one  of  which  is  system  latency  measured  by  the  end-to-end  time  taken  for  a  system 
to  react  to  a  user’s  command.  Another  important  factor  is  to  gauge  the  throughput  of  the  sys¬ 
tem,  measuring  the  bytes  per  second  processed  by  the  system.  Benchmarking  the  distributed 
MYSEA  system  will  help  to  reveal  potential  design  inefficiencies  so  that,  in  the  long  run,  steps 
can  be  taken  to  improve  its  performance  and  efficiency.  A  bottleneck  occurs  when  the  perfor¬ 
mance  of  a  system  is  limited  by  a  number  of  components  or  resources,  thus  reducing  the  overall 
throughput  or  increasing  the  latency  experienced  by  a  user.  In  the  absence  of  bottlenecks,  our 
secure  system  performance  study  can  be  interpreted  as  characterizing  the  “cost  of  security”  in 
a  multilevel  security  context. 

1.3  Thesis  Organization 

The  remainder  of  this  thesis  is  organized  as  follows:  Chapter  2  provides  the  background  on 
MYSEA  and  an  overview  of  its  trusted  components  and  services,  which  in  combination  form 
the  multilevel  secure  distributed  system.  It  also  provides  a  detailed  discussion  of  the  design  of 
MYSEA  and  the  design  considerations  that  impact  its  performance.  Chapter  3  outlines  the  ob¬ 
jectives  of  the  thesis  by  identifying  the  high-level  and  detailed  requirements  for  benchmarking 
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MYSEA,  including  the  criteria  for  selection  of  a  benchmark  tool.  These  requirements  provide 
the  basis  for  the  selection  of  the  KPIs  that  were  measured.  This  chapter  also  analyses  the  various 
benchmark  tools  for  consideration;  compares  the  pros  and  cons  of  each  tool  and  selects  a  tool 
to  benchmark  MYSEA.  It  describes  the  benchmark  methodology  used  to  ensure  that  tests  were 
conducted  fairly,  accurately  and  without  bias.  Chapter  4  describes  the  test  environment,  its  test 
configurations  and  the  test  plan  used  for  the  execution  of  the  benchmarks.  Chapter  5  deals  with 
the  actual  implementation  of  the  benchmarks  and  analyzes  the  results  collected  by  the  various 
benchmark  tests.  Chapter  6  concludes  by  summarizing  the  results,  discussing  related  work  and 
providing  recommendations  for  future  work. 
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CHAPTER  2: 

Monterey  Security  Architecture  Overview 


MYSEA,  The  Monterey  Security  Architecture,  is  a  distributed  client-server  architecture  com¬ 
posed  of  existing  Government-off-the-shelf  (GOTS)  and  Commercial-off-the-shelf  (COTS)  soft¬ 
ware,  open-source  components,  and  (relatively  few)  special-purpose,  high-assurance  compo¬ 
nents  [1]  [2]  [3]  [4],  MYSEA  provides  a  trusted  computing  operating  environment  for  enforcing 
information-flow  policies  in  a  multilevel  LAN.  MYSEA  allows  vertical  integration  of  applica¬ 
tion  security  requirements  with  underlying  security  services;  an  existing  Quality  of  Security 
Service  model  [5]  and  framework  are  applied  to  the  integrated  security  structure.  In  addition, 
MYSEA  supports  a  secure  and  unforgeable  bidirectional  connection,  called  a  trusted  path,  be¬ 
tween  the  user  and  the  trusted  elements  of  the  systems  of  the  system,  as  well  as  trusted  channels 
between  devices. 

MYSEA  enforces  confidentiality  and  integrity  policies  formalized  in  the  Bell  and  La-Padula 
(BLP)  and  Biba  security  policy  models  respectively.  The  high  assurance  MYSEA  Servers  en¬ 
force  the  security  policies  and  control  the  information  that  a  user  can  access  at  a  particular  sensi¬ 
tivity  level.  These  servers  host  various  open-source  or  commercial  application  servers  (i.e.,  web 
server  and  mail  server),  providing  services  along  with  multilevel  views  of  information  to  clients. 
MYSEA  currently  supports  Simple  Mail  Transfer  Protocol  (SMTP)  [6],  Internet  Message  Ac¬ 
cess  Protocol  (IMAP)  [7],  Voice  over  Internet  Protocol  (VoIP)  [8],  Web  Distributed  Authoring 
and  Versioning  (WebDAV)  [9],  Hypertext  Transfer  Protocol  (HTTP)  [10]  and  Wiki  [11]. 

MYSEA  clients  are  commodity  PCs  equipped  with  specialized  Trusted  Path  Extension  (TPE) 
devices.  TPEs  are  inserted  between  unsecured  client  workstations  and  the  MYSEA  server. 
Every  network  connection  from  a  MYSEA  client  to  the  Multilevel  Security  (MLS)  servers  is 
labeled,  thus  permitting  the  enforcement  of  the  security  policies  for  data  entering  MYSEA 
Server.  Together,  the  MYSEA  Servers  and  TPEs  form  the  Trusted  Computing  Base  (TCB). 

Section  2.1  provides  the  MYSEA  design  overview  and  Section  2.2  describes  a  typical  usage 
scenario  of  how  clients  gain  access  to  the  services  hosted  on  MYSEA  server;  Section  2.3  pro¬ 
vides  a  detailed  description  of  the  important  components  and  services  in  MYSEA  and  possible 
sources  of  performance  overhead.  Section  2.4  outlines  the  security  policy  enforced  in  MYSEA 
and  related  design  decisions.  Section  2.5  describes  the  current  MYSEA  prototype. 
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2.1  MYSEA  Design  Overview 

MYSEA  is  designed  to  provide  application  services  in  a  high  assurance,  distributed  environ¬ 
ment,  while  protecting  users  from  malicious  code  and  other  attacks.  The  design  of  MYSEA 
allows  mandatory  policy  enforcement  mechanisms  to  be  allocated  to  a  few  specialized  high- 
assurance  trusted  components,  while  encompassing  other  low  assurance  commercial  compo¬ 
nents  and  applications  to  support  operational  functionality. 

The  key  design  principles  of  MYSEA  are  (1)  security  dependencies  must  be  partially  ordered; 
(2)  the  ordering  of  trust  and  trustworthiness  must  be  present  and  (3)  the  principle  of  least  priv¬ 
ilege  must  be  enforced.  A  federation  of  highly  trustworthy  MLS  servers  provides  the  locus  of 
policy  enforcement,  mediating  all  accesses  to  resources  in  the  system.  The  MYSEA  servers 
host  various  open-source  or  commercial  services  such  as  mail  and  web  services.  The  MYSEA 
federation  is  designed  to  be  a  robust  and  scalable  foundation  supporting  expansion:  the  scala¬ 
bility  requirement  for  MYSEA  is  to  support  up  to  100  clients  with  concurrent  user  sessions  at 
100  different  security  levels. 

To  establish  a  secure  communication  channel  between  clients  and  MYSEA  servers,  trusted 
TPEs  authenticate  and  disambiguate  clients.  All  network  traffic  flows  in  and  out  of  MYSEA 
are  tracked  to  determine  the  security  level  of  the  client  making  the  request.  This  results  in  an 
implicit  labeling  of  all  network  traffic. 

MYSEA  servers  and  TPEs  collectively  enforce  the  mandatory  security  policies,  forming  a 
Trusted  Computing  Base  (TCB).  The  MYSEA  TCB  is  designed  to  facilitate  evaluation  by 
TCB  subsets  [12].  TCB  subsets  allow  for  the  partition  of  a  complex  system  into  smaller  dis¬ 
joint  trusted  subsets,  each  of  which  resides  in  its  own  protection  domain  and  enforces  a  security 
policy  subset  upon  the  subjects  and  objects  under  its  control  [12].  The  composition  of  each 
TCB  subset  is  small  and  less  complex,  and  can  be  evaluated  individually  for  policy  enforce¬ 
ment  correctness.  This  chain  of  incremental  evaluations  of  TCB  subsets  combined  with  domain 
mechanism,  which  in  the  case  of  MYSEA  is  both  logical  and  physical,  ensures  that  the  policies 
are  enforced. 


2.2  MYSEA  Concept  of  Operations 

Users  logon  to  the  MYSEA  server  using  a  Single  Level  At  a  Time  (SLAT)  client  workstation. 
The  commodity  workstations  can  be  on  the  local  multilevel  LAN  as  well  as  on  legacy  single 
level  networks.  Clients  on  the  multilevel  LAN  must  have  an  associated  TPE.  Labels  for  clients 
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on  legacy  networks  are  implicit,  as  all  clients  are  at  the  level  of  the  network  itself.  Each  LAN- 
based  client  workstation  and  TPE  is  logically  and  physically  bound  together.  There  is  exactly 
one  TPE  for  each  client  workstation,  and  no  more  than  one  user  at  a  time  can  be  active  on  the 
given  TPE.  The  TPE  distinguishes  user  sessions  and  establishes  a  trusted  path  of  communica¬ 
tion  to  the  MYSEA  server.  Upon  logon,  the  TPE  establishes  an  identity  for  audit  and  access 
control  purposes;  and  forwards  the  user’s  credentials  to  MYSEA  server. 

Next,  the  user  uses  the  TPE  to  negotiate  a  session  level  within  his  security  clearance  range. 
Based  on  the  security  policy  enforced  by  the  MYSEA  server,  the  session  level  determines  the 
domain  and  resources  he  can  access  during  that  session.  For  example,  to  access  resources  at  a 
higher  classification  level,  the  user  would  need  to  change  his  session  level  using  the  TPE. 

Following  session  level  negotiation,  the  SLAT  client-TPE  pair  can  be  used  to  access  applications 
hosted  on  any  MYSEA  server  or  on  a  single-level  network  at  the  user’s  session  level.  When 
activity  at  a  particular  session  level  is  over,  data  created  or  modified  on  the  client  is  stored  on 
the  MYSEA  server,  and  the  SLAT  client  state  is  automatically  purged  at  the  end  of  each  session. 


Figure  2.1:  Major  Components  in  MYSEA  From  [1] 
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2.3  Overview  of  MYSEA  Components  and  Services 

This  section  describes  the  major  MYSEA  components  and  services  that  comprise  MYSEA’s 
design.  Figure  2.1  shows  the  relationship  between  some  major  components  and  services  in 
MYSEA.  The  major  components  in  MYSEA  include: 

1 .  Clients  and  TPEs.  Each  SLAT  client  executes  popular  software  applications  and  pro¬ 
ductivity  tools  such  as  word  processors  and  web  browsers.  The  TPEs  controls  the  in¬ 
terface  between  the  client  and  the  MYSEA  server.  The  TPE  provides  transport-layer 
security,  trustworthy  identification  and  authentication,  and  session  establishment. 

2.  MYSEA  Trusted  Proxy  Service.  Various  untrusted,  open  source  applications  hosted 
on  the  MYSEA  server  provide  services  (e.g.  web  services  and  mail  services)  to  clients. 
The  clients  cannot  directly  access  these  applications.  Instead,  clients  invoke  the  services 
through  a  layer  of  proxy  processes  provided  by  MYSEA. 

3.  MYSEA  server  Federation.  The  MYSEA  architecture  is  distributed,  leveraging  a 
cluster  of  high-assurance  MYSEA  servers.  These  servers  provide  the  locus  of  multilevel 
security  policy  enforcement  and  host  various  open  source  or  commercial  application  pro¬ 
tocol  servers. 

4.  Dynamic  Security  Services.  The  Dynamic  Security  Services  (DSS)  manages  the 
transport-layer  security  used  to  protect  the  trusted  path  between  the  TPE  and  remote 
MYSEA  server. 

5.  Single-Level  Networks.  Existing  classified  single  level  networks  are  connected  to  the 
MYSEA  server  providing  application  services  to  both  local  clients  and  clients  on  legacy 
networks. 

2.3.1  Clients  and  TPES. 

MYSEA  clients  are  SLAT,  and  can  only  access  and  process  data,  at  a  single  session  level  at  one 
time.  MYSEA  clients  consist  of  two  components:  commodity  PCs  and  TPEs.  Commodity  PCs 
are  untrusted,  “thin-client”  machines,  running  familiar  commercial  software  and  OSes.  They 
are  loaded  with  COTS  products  and  open-source  software  that  assist  the  user  to  perform  tasks 
like  word-processing,  web  browsing  and  communication.  With  this  software,  users  can  access 
remote  applications  hosted  on  a  MYSEA  server  or  single-level  network. 

The  TPEs  operate  largely  under  the  direction  of  the  remote  MYSEA  server  and  help  to  enforce 
MLS  LAN  policies.  All  network  traffic  in  and  out  of  a  SLAT  Client  is  routed  through  its  TPE. 
The  TPE  ensures  that  a  secure  tunnel  is  established  between  client  and  TCB.  This  tunnel  is 
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encrypted  with  Internet  Protocol  Security  (IPSec)  leveraging  DSS  services  (see  Section  2.3.4). 

When  a  remote  session  is  established,  the  TPE  sends  a  request  for  a  service  via  the  secure  tunnel 
to  the  MYSEA  server.  The  TPE  acts  as  a  Network  Address  Translation  (NAT)  device,  perform¬ 
ing  address  translation  on  all  traffic  traversing  between  the  client  and  the  MYSEA  server.  On 
session  establishment,  the  TPE  passes  the  user’s  credentials  to  the  MYSEA  server,  which  val¬ 
idates  the  credentials  and  instructs  the  TPE  whether  to  allow  or  deny  access.  The  user  may 
use  the  TPE  to  send  commands  to  the  MYSEA  server  to  perform  session-related  tasks  such  as 
ending  the  session  or  changing  session  level. 

2.3.2  MYSEA  Trusted  Proxy  Service 

Each  MYSEA  server  consists  of  a  high  assurance  operating  system  that  enforces  the  mandatory 
security  policy.  The  high  assurance  kernel  creates  labeled  protected  domains  and  associates 
security  attributes  with  active  and  passive  entities  exported  at  its  interface  [4],  This  results  in 
a  distinct  separation  of  data  into  different  security  classification  domains.  In  MYSEA,  trusted 
multilevel  processes  control  the  invocation  of  single-level  applications  that  interact  with  the  user 
at  his  current  session  level.  Various  trusted  processes  are  created  by  a  daemon  during  system 
initialization  and  are  responsible  for  the  proxying  and  mediating  network  access  on  behalf  of 
the  application  service,  allowing  it  to  communicate  with  the  remote  client. 

For  every  application  service  hosted  on  MYSEA,  a  Secure  Session  Server  (Parent)  (SSS-P)  is 
created  to  respond  to  connection  requests  from  clients.  The  SSS-P  monitors  the  network  and  is 
responsible  for  accepting  and  verifying  that  service  requests  from  a  TPE  have  a  valid  session. 
The  SSS-P  spawns  a  proxy  service  (Secure  Session  Server  (Child)  (SSS-C))  for  each  client 
connection  request.  This  proxy  sets  up  the  actual  untrusted  Application  Protocol  Service  (APS) 
associated  with  the  request,  e.g.  an  untrusted  Apache  web  service  is  an  APS  at  the  level  of  the 
remote  user’s  session.  All  subsequent  communication  between  the  client  and  the  APS  is  sent 
and  received  via  the  proxy  SSS-C,  using  one  or  more  “MYSEA  sockets”. 

2.3.3  MYSEA  Server  Federation 

The  MYSEA  server  federation  is  a  cluster  of  high  assurance  servers  that  provide  the  locus  of 
multilevel  security  policy  enforcement  and  hosts  various  open  source  or  commercial  services. 
During  session  establishment,  the  TPE  forwards  the  user’s  credentials  to  the  MYSEA  server 
hosting  the  Trusted  Path  Server  (TPS)  process.  The  TPS  maintains  session  data  including  the 
user  credentials,  user’s  clearance  range  and  his  current  session  level.  The  TPS  process  is  respon- 
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sible  for  validating  connection  requests  from  the  TPE,  and  handles  TPE  requests  for  updating 
session  information  (like  login,  change  session  level,  run  and  logout),  ensuring  that  account¬ 
ability  aspects  of  the  security  policy  are  enforced. 

In  the  MYSEA  server  federation,  a  client  can  request  an  APS  service  hosted  on  a  server  that  is 
not  hosting  the  TPS  process.  A  Federated  Services  Daemon  (FSD)  on  each  MYSEA  server  han¬ 
dles  the  internal  communication  required,  validating  the  authenticity  of  the  client  and  mediating 
the  accesses  to  the  requested  resources  according  to  the  security  policies. 

2.3.4  Dynamic  Security  Services 

DSS  is  an  IPSec  configuration  and  administrative  tool  that  assists  in  the  creation  of  secure 
tunnels  between  TPEs  and  the  MYSEA  server.  Each  DSS  client  resides  on  the  TPE  and  connects 
to  the  DSS  server  on  the  MYSEA  server.  The  DSS  admin  tool  allows  a  security  administrator 
to  log  onto  the  MYSEA  server,  set  and  manage  the  security  policies  associated  with  the  IPSec 
tunnels. 


2.3.5  Single-level  Networks 

Users  on  a  legacy  network  can  only  access  applications  on  the  MYSEA  server  at  the  classifi¬ 
cation  level  associated  with  the  label  of  the  corresponding  network  device.  Similarly,  MYSEA 
clients  can  also  access  applications  on  the  legacy  networks.  This  allows  existing  applications 
on  legacy  networks  to  run  unmodified  and  be  accessible  to  SLAT  client  on  the  MLS  LAN,  with 
the  MYSEA  TCB  mediating  all  accesses. 

2.4  MYSEA  Security  Policies 

MYSEA  supports  both  Mandatory  Access  Control  (MAC)  and  Discretionary  Access  Control 
(DAC)  [1].  The  system  MAC  policy  is  a  confidentiality  and  integrity  information  flow  control 
policy,  where  the  sensitivity  labels  form  a  lattice  [13].  The  confidentiality  and  integrity  policies 
are  formally  modeled  by  the  BLP  [14]  and  Biba  [15]  models,  respectively.  The  high-assurance 
OS  of  the  MYSEA  server  mediates  access  to  files  and  other  labeled  system  resources.  Network 
connections  on  the  MLS  LAN  are  mediated  by  small,  trusted  processes  on  the  MYSEA  Server, 
using  information  about  client  session  level.  For  example,  a  client  at  SECRET  cannot  com¬ 
municate  with  a  client  at  UNCLASSIFIED,  nor  can  a  client  at  SECRET  communicate  using  a 
port  in  use  by  an  untrusted  APS  acting  on  behalf  of  a  remote  client  at  UNCLASSIFIED.  Thus, 
access  to  network  resources  is  impacted  by  checks  performed  by  MYSEA  processes,  and  access 
to  system  resources  is  impacted  by  checks  performed  by  the  underlying  high- assurance  OS. 
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For  Discretionary  Access  Control  (DAC),  the  MYSEA  server  allows  users’  processes  to  modify 
permissions  associated  with  data  objects  such  as  files  via  runtime  functions.  The  permissions 
are  registered  with  the  server,  and  access  decisions  are  made  based  on  the  end-users’  identities 
bounded  to  the  processes.  This  policy  is  less  critical  than  MAC;  hence  its  enforcement  can  be 
allocated  to  the  applications  hosted  on  MYSEA  server.  Users  can  determine  the  access  controls 
of  APS  processes  to  access  resources  such  as  files.  The  access  decisions  are  based  on  the 
MYSEA  user  identities  associated  with  the  processes  and  permissions  that  are  registered  with 
MYSEA  server.  This  policy  can  be  modified  during  runtime. 

2.5  Current  MYSEA  Prototype 

To  benchmark  MYSEA,  we  will  analyze  the  existing  MYSEA  prototype  system.  Here,  we  high¬ 
light  some  consequential  differences  between  the  current  prototype  system,  and  the  MYSEA 
design.  Our  benchmark  tests  will  be  performed  on  the  current  prototype. 

2.5.1  Clients  and  TPEs 

There  are  several  designs  capable  of  satisfying  the  MYSEA  client  component.  The  TPE  ap¬ 
plication  and  SLAT  client  could  both  run  on  a  trusted  separation  kernel  or  hypervisor:  each 
single-level  client  runs  in  its  own  partition;  the  TPE  application  runs  in  its  own  partition.  Alter¬ 
natively,  these  components  could  be  implemented  using  a  small,  handheld  tactical  device  that 
connects  the  stateless,  thin  client  to  the  MLS  LAN.  The  current  prototype  system  follows  this 
latter  strategy:  the  TPE  mediates  access  to  the  network  for  a  stateless  client  (currently  a  com¬ 
modity  PC  that  must  be  manually  purged  of  state  whenever  a  session  ends).  The  prototype  TPE 
application  is  hosted  on  Linux,  running  on  a  hand-held  device. 

2.5.2  MYSEA  Server  Federation 

Each  MYSEA  server  is  designed  to  run  on  an  existing,  commercial  high-assurance  system. 
Previous  MYSEA  prototypes  utilized  BAE  XTS-400  running  STOP  6,  an  operating  system  with 
CC  EAL5+  certification  [16].  The  current  prototype  uses  the  next  generation  of  this  product: 
STOP  7,  which  attained  a  CC  EAL4+  certification  [17]. 

2.5.3  Dynamic  Security  Services 

The  security  requirements  of  MYSEA  necessitate  the  establishment  of  a  secure  tunnel  for  net¬ 
work  communication  between  the  TPE  and  MYSEA  server.  As  STOP  7  lacks  IPsec  support, 
another  mechanism  for  establishing  a  secure  tunnel  between  the  client  and  server  is  needed. 
A  host-to-gateway  IPSec  tunnel  is  established  between  the  TPE  and  a  Server  Gateway  device. 
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All  network  traffic  to  the  MYSEA  server  is  routed  through  the  Server  Gateway.  Together,  the 
MYSEA  server  and  Server  Gateway  form  a  single  logical  unit.  The  DSS  client  on  both  the  TPE 
and  Server  Gateway  control  this  IPsec  tunnel,  modulating  the  tunnel’s  characteristics  based  on 
requests  from  a  remote  DSS  admin  tool.  The  DSS  Client  uses  Racoon  [18],  the  user-space 
IPSec  daemon,  to  provide  the  dynamic  protection  services. 
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CHAPTER  3: 

Objectives  and  High-Level  Analysis 


This  chapter  outlines  the  factors  that  are  considered  during  the  conceptual  and  planning  phases 
for  benchmarking  MYSEA  as  well  as  details  about  the  selection  of  the  tool  used  for  performance 
measurement. 

We  are  interested  in  the  overhead  introduced  by  MYSEA’s  design,  such  as  those  introduced 
by  processes  performing  Mandatory  Access  Control  (MAC)  checks  associated  with  network 
related  calls.  We  attempt  to  measure  components  individually  to  determine  which  act  as  system 
bottlenecks.  By  isolating  and  measuring  the  behavior  of  individual  components,  rather  than 
measuring  the  overall  system  or  application  performance,  we  are  able  to  quantitatively  assess 
MYSEA’s  design.  Figure  3.1  outlines  the  process  flow  resulting  from  a  client’s  requests  to 
servers  in  MYSEA. 


Figure  3.1:  Process  flow  of  a  client  making  a  service  request  to  MYSEA 


We  describe  the  events  that  ensue  next. 
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1.  Establish  IPSec  Tunnel 

The  DSS  client  on  the  TPE  establishes  an  IPSec  tunnel  with  the  Server  Gateway  for  the 
MYSEA  server,  forming  a  trusted  communications  path  between  the  client  and  MYSEA. 
The  DSS  client  on  the  TPE  operates  as  a  slave,  following  the  DSS  policy  dictated  by  the 
MYSEA  server. 

We  can  measure  the  overhead  introduced  as  every  network  packet  is  inspected  and  an¬ 
alyzed  according  to  the  loaded  IPSec  policy.  If  the  communication  is  allowable  by  the 
security  policy,  every  network  packet  is  encrypted  at  layer  three  of  the  TCP/IP  protocol 
stack  and  forwarded  to  the  MYSEA  Server  via  the  secure  communications  channel. 

2.  Login  and  Start  Secure  Session 

Once  an  IPSec  tunnel  has  been  established,  the  user  logs  on  to  MYSEA  server  using 
the  TPE  and  establishes  a  session  at  some  security  level.  The  TPS  process  on  MYSEA  is 
responsible  for  identification  and  authentication  (I&A)  and  must  monitor  for  TPE  connec¬ 
tion  requests.  There  is  a  TPS  Parent  process  for  the  MYSEA  federation,  which  manages 
all  Identification  and  Authentication  (I&A)  and  login  requests.  The  TPS  Parent  accepts 
login  requests  from  MYSEA  clients  and  spawns  a  TPS  Child  to  handle  trusted  path  com¬ 
munications  for  each  client.  When  the  TPS  child  receives  data  from  the  client,  it  reads 
and  processes  the  client’s  command  and  returns  the  output  before  waiting  to  receive  the 
next  command.  Upon  the  successful  I&A,  the  user  at  the  MYSEA  client  negotiates  a 
session  within  his  security  range  to  access  the  services  hosted  on  the  MYSEA  server. 

The  user  can  issue  a  “Run”  or  “Logout”  command  via  the  TPE.  The  latter  terminates 
the  secure  session  and  the  TPS  on  MYSEA  takes  actions  to  update  its  information  about 
active  sessions.  Various  interactions  can  take  place  during  the  login  process,  resulting  in 
overhead  impacting  the  performance  of  MYSEA. 

3.  Service  Request 

Every  service  request  made  from  the  client  to  the  APS  goes  through  a  series  of  proxy  calls 
before  it  is  processed  by  the  APS.  The  APS  is  the  application  process  that  responds  to  the 
service  request,  and  cannot  directly  communicate  with  the  client.  The  APS  is  an  untrusted 
process;  the  SSS-C  process  mediates  all  network  connections.  Communications  between 
the  APS  and  SSS-C  are  handled  through  a  MYSEA  socket  (MSKT)  allocated  by  the  TPS 
Child  process.  When  the  APS  requires  a  socket  call,  the  APS  updates  the  MSKT  database, 
and  signals  the  SSS-C.  The  SSS-C  retrieves  the  request  from  the  database,  responds  to  the 
client’s  service  request  and  inserts  any  retrieved  data  into  the  database.  The  APS  is  then 
notified  to  retrieve  the  requested  data  from  the  database.  Each  TCP  connection  (for  UDP, 
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each  packet)  is  checked  using  its  source  and  destination,  to  see  (in  the  context  of  active 
sessions  and  outstanding  network  use)  if  the  information  flow  is  allowable  according  to 
the  MAC  policy.  When  the  APS  and  TPS  processes  are  deployed  on  different  physical 
servers,  there  may  be  additional  overhead  due  to  the  communication  between  MYSEA 
servers,  querying  about  the  status  of  active  sessions  within  the  Federation.  These  proxy 
design  decisions  may  impact  the  performance  of  MYSEA. 

We  are  interested  in  measuring  the  performance  impact  of  the  following  three  components:  (1) 
IPSec,  (2)  the  MYSEA  trusted  proxy  and  (3)  MYSEA’s  federation  of  servers  as  a  result 
of  the  events  described  earlier.  IPSec  is  related  to  the  cost  of  IPSec  encryption  of  all  network 
traffic  between  the  TPE  and  server  gateway.  Measuring  the  MYSEA  trusted  proxy  allows  us 
to  analyze  the  cost  of  the  client  making  a  service  request  to  MYSEA.  We  can  measure  the 
overhead  associated  with  the  proxying  of  system  calls  for  making  any  service  request.  Targeting 
the  MYSEA  federation  allows  us  to  measure  the  cost  of  maintaining  state  among  the  servers. 

The  output  of  a  benchmark  must  be  reproducible,  consistent,  unbiased  and  realistic,  based  on 
sound  scientific  methods  and  principles.  The  demands  for  a  successful  benchmark  imply  a  need 
for  proper  scientific  procedure  and  a  need  for  common  agreement  on  the  metrics  [19].  There¬ 
fore,  the  benchmarking  methodology  adopted  in  this  thesis  consists  of  a  four  stage  process:  (1) 
determine  the  performance  indicators;  (2)  select  or  develop  tools  for  the  benchmark;  (3)  config¬ 
ure  the  test  environment  and;  (4)  execute  benchmarks  and  analyze  output.  We  describe  stages 
(1)  and  (2)  in  this  chapter,  stage  (3)  in  Chapter  4  and  stage  (4)  in  Chapter  5. 


3.1  Benchmarking  Methodology 

Benchmarking  is  the  process  of  comparing  business  processes,  services  or  products  against  one 
another.  It  allows  organizations  to  understand  quantitatively  the  performance  of  the  targets, 
possibly  for  the  purpose  of  improving  their  characteristics.  Benchmarking  of  an  IT  system  is 
the  process  of  running  a  specific  program  or  workload  on  a  specific  machine  or  system  and 
measuring  the  resulting  performance  [20]. 

There  are  a  wide  variety  of  approaches  to  benchmarking  because  different  approaches  are  neces¬ 
sary  to  evaluate  individual  aspects  of  a  system.  For  example,  synthetic  benchmarks  load  or  stress 
test  some  specific  targeted  components  in  the  system;  application  benchmarks  measure  perfor¬ 
mance  of  the  system  as  it  accomplishes  some  real-world  task  (like  running  a  web  server)  under 
some  synthetic  load.  Benchmarks  can  be  divided  into  the  following  categories  [19]  [21]  [22]: 
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•  Application  benchmarks,  also  known  as  macro  benchmarks,  take  an  application  and 
measure  the  time  required  to  complete  some  tasks.  They  measure  end-to-end  perfor¬ 
mance  on  a  specific  workload,  assessing  the  performance  experienced  by  end-users.  For 
example,  the  library  generator  ATLAS  [23]  uses  application  benchmarking  to  build  linear 
algebra  libraries  automatically  tuned  for  the  target  processor.  An  evaluation  of  the  Denali 
Isolation  kernel  [24]  made  use  of  web  server  benchmarks  to  compare  that  platform  with 
BSD.  The  SPEC  CPU  benchmark  suite  measures  several  applications  and  combines  the 
results  to  rank  overall  performance. 

•  Micro-benchmarks  measure  the  performance  of  individual  operations  in  isolation.  Bench¬ 
marks  intended  to  measure  the  performance  of  kernel  primitives;  system  calls  and  the 
network  stack  are  commonly  measured  using  micro-benchmarks.  Ousterhout  [25]  de¬ 
veloped  benchmarks  to  isolate  a  number  of  kernel  primitives  providing  measurements  of 
performance,  independent  of  the  machine’s  platform.  Micro-benchmarks  were  also  used 
to  quantify  the  performance  of  the  Denali  Isolation  Kernel’s  primitive  operations  [24], 
Network  micro  benchmarks  measure  the  bandwidth,  throughput  and  network  latency  ex¬ 
perienced  by  the  system,  irrespective  of  application.  Such  benchmarks  provide  insight 
into  low-level  network  stack  performance;  in  particular  the  latency  associated  with  open¬ 
ing  a  socket  connection.  A  performance  characterization  of  the  Chelsio  T110  10-Gigabit 
Ethernet  [26]  adaptor  makes  use  of  micro-benchmarks  to  measure  the  single  and  multiple 
connections  of  point-to-point  latency  and  unidirectional  throughput. 

•  Other  benchmarks  target  the  hardware  used  by  the  system,  or  the  end-users  of  the  sys¬ 
tem.  Benchmarking  I/O  performance  provides  a  measurement  of  the  cost  of  retrieving  or 
storing  data  using  some  particular  hardware.  It  may  measure  sequential  and  streaming  I/O 
bandwidth  for  single  processes,  or  random  I/O  latency  for  multiple  processes  (including 
the  time  taken  to  create,  write  and  read  a  number  of  files  in  a  file  system). 

Subjecting  real-world  users  to  the  system  provides  realistic  performance  measures.  These 
user  tests  produce  results  that  are  not  always  comparable  across  systems  or  versions,  as 
these  tests  cannot  be  easily  repeated. 

3.2  Performance  Indicators 

Different  tools  measure  different  performance  indicators  such  as  processor  speed,  memory  la¬ 
tency,  network  throughput,  bandwidth  and  socket  connection  latency.  Chapter  2  describes  the 
MYSEA  process  structure  and  their  relationships.  For  a  user  to  access  an  APS  service  hosted  on 
MYSEA,  secure  communication  is  established  between  the  MYSEA  client  and  MYSEA  server 
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using  IPSec  encryption.  Every  network  packet  is  inspected  and  encrypted  at  the  TPE  before 
it  is  transmitted  to  the  server  gateway.  At  the  server  gateway,  network  packets  are  decrypted 
and  forwarded  to  the  MYSEA  server  for  processing.  Also,  during  the  initial  setup  of  APS  com¬ 
munication,  the  SSS-P  listens  on  the  network  for  requests  and  spawns  a  proxy  to  handle  the 
connection.  The  proxy  acts  as  an  intermediary  between  the  APS  and  network  to  handle  the 
request.  All  traffic  is  inspected  and  proxied  within  MYSEA  before  the  APS  can  receive  it.  In 
addition,  the  TPS  process  that  handles  all  I&A  can  be  deployed  on  a  different  server  from  those 
hosting  MYSEA’s  APS  services.  If  a  service  request  is  made  to  an  APS  hosted  on  a  server  other 
than  the  TPS,  internal  communication  between  the  servers  is  required  to  assert  the  identity  and 
authenticity  of  the  request.  The  following  two  KPIs  are  identified  to  measure  the  performance 
of  MYSEA.  First,  we  can  measure  the  TCP  and  UDP  throughput  transfer  rates  of  the  IPSec 
encryption,  MYSEA  proxy  and  MYSEA  federation.  Second,  we  can  also  measure  network 
latency,  the  time  taken  for  data  to  be  transmitted  from  one  point  to  another.  In  MYSEA,  one 
could  measure  the  latency  associated  with  opening  and  closing  a  network  socket,  reflecting  the 
initial  costs  of  the  setup  for  a  connection  between  the  client  and  server.  We  can  also  measure 
the  latency  associated  with  the  time  taken  for  a  request  and  response  transaction  to  complete. 

3.3  Benchmarking  Tools  for  Use  in  MYSEA 

There  are  various  benchmarking  tools  available  to  measure  the  performance  of  systems  that 
provide  networked  services  such  as  MYSEA.  Examples  include  Apache  Jmeter,  LMbench  and 
Netperf.  These  benchmark  tools  automate  the  process  of  running  performance  tests  for  the 
targeted  system.  These  tools  are  intended  to  produce  repeatable  results,  allowing  for  consistent 
analysis  of  data  to  compare  system  performance. 

Any  benchmark  tools  used  in  the  context  of  MYSEA  must  satisfy  certain  operational  require¬ 
ments.  The  current  MYSEA  prototype  runs  on  the  STOP  operating  system,  a  proprietary  high 
assurance  operating  system  that  is  binary  compatible  with  Linux.  Any  application  service 
hosted  on  MYSEA  needs  to  be  modified  to  run  in  a  specific  inetd  mode,  integrated  with  MY¬ 
SEA  processes.  Therefore,  source  code  for  the  tool  selected  must  be  available,  and  compatible 
in  the  MYSEA  environment. 

The  tool  selected  must  be  able  to  measure  specific  targets  of  the  MYSEA  system.  We  desire  to 
characterize  each  MYSEA  component  to  analyze  each  feature  of  MYSEA’s  design.  In  particu¬ 
lar,  an  end-to-end  application  benchmark  tool  will  not  be  able  to  produce  such  measurements, 
as  it  characterizes  the  system  only  in  terms  of  its  performance  with  a  specific  application.  For 
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example,  MYSEA  currently  supports  a  variety  of  application  services  (web  services,  VoIP,  etc.), 
and  the  benchmark  tool  selected  must  not  characterize  the  performance  of  the  application  ser¬ 
vice.  Additionally,  as  MYSEA  is  an  experimental  system  without  actual  users,  we  are  unable  to 
develop  workloads  representative  of  its  use  in  a  realistic  operating  environment.  The  following 
sections  described  the  benchmark  tools  considered. 

3.3.1  Jmeter 

Apache  Jmeter  (TM)  [27]  is  an  open-source  web  server  benchmark  written  in  Java  designed  to 
test  functional  behavior  and  measure  performance.  It  can  be  used  to  test  performance  on  static 
and  dynamic  web  resources,  and  perform  load  testing  on  the  server.  It  is  extensible  and  allows 
the  development  of  plugins  that  provides  extensible  testing  capabilities. 

Jmeter  can  be  configured  to  benchmark  the  performance  of  the  existing  web  services  on  MYSEA 
at  the  client.  This  would  eliminate  the  need  to  modify  or  adapt  the  tool  to  run  on  MYSEA.  How¬ 
ever,  Jmeter  is  unable  to  produce  results  that  measure  isolated  components  of  MYSEA,  for  ex¬ 
ample  the  cost  of  proxying  specific  network  calls.  This  tool  would  characterize  the  performance 
costs  of  MYSEA’s  TCB,  combined  with  the  performance  characteristics  of  the  un trusted,  third 
party  web  service,  rather  than  characterizing  the  MYSEA  system  performance  in  isolation. 

3.3.2  Netperf 

Netperf  [28]  is  an  open-source  benchmark  tool  written  in  C  that  is  designed  to  characterize  gen¬ 
eral  network  performance  experienced  in  client-server  settings.  The  tool  is  split  into  two  com¬ 
ponents:  netperf  and  netserver.  The  netperf  tool  is  a  client  that  requests  tests  to  be  performed, 
while  the  netserver  tool  is  a  daemon  process  that  resides  on  the  system  under  test  and  responds 
to  requests.  Netserver  can  run  as  either  as  a  daemon,  or  as  an  inetd  service.  Netperf  provides 
tests  for  both  unidirectional  and  bidirectional  throughput  and  end-to-end  latency.  It  can  be  used 
to  measure  various  aspects  of  networking  performance  with  primary  focus  on  unidirectional 
data  transfer  and  request/response  performance  using  TCP  or  UDP  socket  interfaces. 

3.3.3  LMbench 

LMbench  [22]  [29]  is  a  suite  of  micro-benchmarks  written  in  C  designed  to  measure  latency 
and  bandwidth  performance  of  system  operations  like  data  movement  among  the  processor 
and  memory,  network,  file  system  and  disk.  LMbench  was  developed  to  identify  and  evaluate 
system  performance  bottlenecks,  and  is  portable  across  a  wide  set  of  Unix  systems.  LMbench  is 
open-source  and  binary  compatible  with  Linux.  It  can  also  be  used  to  measure  specific  targeted 
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components  and  servers.  However,  LMbench  lacks  comprehensive  documentation:  its  manual 
pages  lack  detail,  and  the  documentation  does  not  describe  how  each  specific  test  measures  its 
target.  The  lack  of  documentation  hinders  the  porting  of  LMbench  source  code  to  work  in  the 
MYSEA  environment. 

3.3.4  SPEC 

Standard  Performance  Evaluation  Corporation  (SPEC)  is  a  non-profit  corporation  that  develops 
and  maintains  a  standardized  set  of  relevant  benchmarks  for  measuring  the  performance  of  IT 
systems.  Organizations  can  measure  the  performance  of  their  products  or  services  using  the 
SPEC  benchmark  suites.  The  results  can  be  submitted  to  SPEC  for  validation,  creating  an 
independently  validated  benchmark.  This  provides  a  fair  comparison  between  similar  products 
and  services.  SPEC  suites  of  benchmarks  are  written  in  C,  Java  or  Fortran.  However  the  source 
code  cannot  be  modified  under  its  licensing  rules  to  prevent  the  optimization  of  benchmarks  to 
skew  the  performance  results  positively,  affecting  fair  comparison.  This  limits  the  portability 
requirement  of  the  benchmark  tool. 
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CHAPTER  4: 
Methodology 


This  chapter  describes  the  test  environment,  configurations  and  test  plan  used  to  benchmark 
selected  key  performance  indicators  in  MYSEA.  The  tests  are  designed  to  produce  results  from 
which  unbiased  comparisons  can  be  drawn.  The  benchmarks  have  been  documented  so  that 
they  can  be  repeated  with  the  same  experimental  setup. 

The  test  environment  consists  of  the  physical  setup,  which  includes  the  servers,  network  routers, 
switches,  and  their  interconnection.  The  software  configuration  used  in  the  benchmark  tests 
consists  of  the  server  operating  systems,  client  operating  systems  and  the  MYSEA  components 
and  services. 

We  can  configure  different  setups  that  allow  for  the  measurement  of  targeted  components  (see 
Chapter  3)  of  the  MYSEA  server.  The  setups  are  formed  by  combining  permutations  of  the  three 
components,  resulting  in  a  total  of  eight  different  setups.  We  compare  the  eight  different  setups 
against  one  another.  The  comparisons  of  different  setups  are  referred  to  as  Test  Configuration 
(TC)s  where  each  TC  measures  the  performance  of  the  components  and  services  in  isolation. 

We  can  execute  a  suite  of  benchmark  tests  on  each  TC,  measuring  the  different  aspects  of  perfor¬ 
mance  of  the  isolated  components  and  services.  Netperf  client  executes  the  following  five  tests: 
(1)  TCP_STREAM,  (2)  TCP_CRR,  (3)  TCP_CC,  (4)  TCP_RR  and  (5)  UDP_STREAM. 

We  describe  the  tool  selected  and  modifications  that  enable  the  benchmarking  of  MYSEA  in 
Section  4.1.  We  describe  the  test  environment  for  the  benchmarks  in  Section  4.2,  the  various 
TCs  in  Section  4.3  and  the  suite  of  benchmark  tests  executed  on  MYSEA  server  in  Section  4.4. 
Section  4.5  describes  the  test  plan  for  performing  the  benchmarks. 

4.1  Tools  and  Instrumentation 

Netperf  version  2.5.0  is  selected  as  the  tool  used  to  benchmark  MYSEA.  There  are  three  reasons 
behind  the  selection  of  netperf  as  the  benchmark  tool.  First,  netperf  is  open-source  and,  second, 
it  is  written  in  the  C  programming  language.  The  necessary  modifications  to  the  netperf  source 
code  can  be  made  to  enable  it  to  run  on  the  current  MYSEA  prototype.  Third,  netperf  has 
comprehensive  documentation  that  describes  in  detail  how  each  test  performs  the  benchmark 
and  what  the  test  is  measuring. 
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To  benchmark  MYSEA,  we  deploy  the  netperf  client  on  the  TPE,  and  netserver  on  the  MYSEA 
server.  The  netperf  client  runs  a  suite  of  benchmark  tests,  making  requests  to  the  netserver 
daemon.  Netserver  responds  to  these  requests,  measuring  latency  and  throughput  performance 
and  returns  the  results  to  netperf. 

Netserver  has  two  modes  of  operations:  inetd  and  daemon.  The  inetd  mode  is  used  in  TCs  using 
MYSEA,  since  all  MYSEA  processes  are  started  by  a  trusted  inetd  process.  The  daemon  mode 
is  used  in  TCs  where  MYSEA  is  not  being  measured,  and  it  is  not  natural  to  start  the  process 
via  inetd. 

In  the  inetd  mode  of  operation,  the  MYSEA  trusted  inetd  process  SSS-P  spawns  the  netserver 
process  when  a  request  is  received  on  the  netserver  service’s  listening  port.  Netserver  has  been 
modified  to  act  as  an  untrusted  MYSEA  service  and  thus  uses  the  MYSEA  proxy  service.  This 
allows  us  to  measure  the  proxy  service  performance.  Refer  to  Appendix  A  for  more  details  on 
the  modifications  to  the  netserver  source  code. 

In  the  daemon  mode  of  operation,  one  starts  the  netserver  process  manually  before  testing.  The 
netserver  daemon  process  monitors  the  network,  responding  to  requests  from  the  netperf  client. 
This  mode  of  operation  does  not  require  any  modification  to  the  netserver  source  code. 

Appendix  B  provides  details  on  how  the  netperf  client  and  netserver  server  process  are  installed, 
and  how  the  benchmark  tests  are  executed. 
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4.2  Test  Environment 


MYSEA  1 


Cisco  Catalyst  3560G 


Figure  4.1:  The  MYSEA  benchmark  test  environment  (network  configuration  and  hardware). 


Figure  4.1  shows  the  configuration  and  topology  of  the  multilevel  LAN  used  during  bench¬ 
marking.  All  network  connections  are  one  gigabit  per  sec  (lGbps)  to  reduce  the  possibility  of 
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Table  4.1:  Configurations  of  TPE,  server  gateway  and  MYSEA  server 


Server  Name 

Configuration 

TPE 

512MB  RAM,  VMWare  ESXi  v4.1.0 

Fedora  7  linux  kernel  2.6.23,  IPSec  Tools  vO.6.6 

Server  Gateway 

3GB  RAM,  VMWare  ESXi  v4.1.0 

Fedora,  IPSec  Tools  vO.6.6 

MYSEA  server 

2GB  RAM,  VMWare  ESXi  v4.1.0 

STOP  v7.3.1 

limited  network  bandwidth  affecting  the  results  of  our  benchmarks.  The  configurations  of  the 
TPE,  server  gateway  and  MYSEA  servers  are  provided  in  Table  4.1. 


4.3  Test  Configuration 

Different  TCs  are  designed  to  measure  and  compare  the  performance  the  MYSEA  components 
and  services  in  isolation.  Figure  4.2  compares  all  possible  test  configurations.  The  setups  are 
formed  by  the  combination  of  three  pairs  of  scenarios:  T7“I-”,  “M”/“M-”  and  “F”/“F-”.  Each 
scenario  is  described  next. 

“I”  represents  the  scenario  when  IPSec  is  used  to  encrypt  all  network  traffic  between  the  TPE 
and  the  MYSEA  server  gateway.  “I-”  represents  the  scenario  when  no  network  traffic  is  en¬ 
crypted  in  the  IPSec  tunnel.  This  is  achieved  by  flushing  all  IPSec  rules  from  Racoon.  There 
are  subtle  differences  between  setups  A5,  B6,  C7  and  D8,  although  the  TCs  indicate  that  they 
measure  the  performance  of  IPSec  encryption.  A5  and  B6  measure  the  performance  of  IPSec 
encryption  with  netserver  spawned  in  inetd  mode  in  a  MYSEA  environment  where  netserver  is 
accessed  via  MYSEA  trusted  proxy  calls.  C7  or  D8  is  deployed  as  a  daemon  on  STOP  server. 

“M”  represents  the  scenario  when  the  netperf  request  undergoes  a  series  of  proxy  calls  in 
MYSEA  before  reaching  the  netserver  process,  with  netserver  process  spawned  by  the  MYSEA 
trusted  inetd  process.  A  secure  session  is  first  established  between  the  TPE  and  the  MYSEA 
server  before  netperf  client  executes  the  suite  of  benchmark  tests.  “M-”  represents  the  sce¬ 
nario  when  the  netserver  process  is  running  as  a  daemon  on  STOP  with  no  MYSEA  processes 
running. 
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“F”  represents  the  scenario  when  the  TPS  process  and  the  netserver  process  are  deployed  on 
different  physical  servers  in  the  MYSEA  federation.  This  results  in  additional  internal  network 
communication  between  the  servers  when  netperf  client  executes  the  tests.  “F-”  represents  the 
scenario  when  the  TPS  process  and  our  benchmark  tool  are  deployed  on  the  same  physical 
server. 

For  example,  “A5”  compares  the  two  test  configurations  “I  M  F”  and  “I-M  F”.  This  test  config¬ 
uration  measures  the  overhead  associated  with  IPSec  (“I”  vs  in  the  configuration  where 
MYSEA  services  are  being  used  (“M”)  on  servers  that  require  internal,  federated  requests  (“F”). 
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Setups 


Figure  4.2:  Figure  showing  the  comparisons  of  different  test  configurations.  Test  cases  considered  in 
this  thesis  are  underlined 
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4.4  Types  of  Tests 

Netperf  is  capable  of  running  different  types  of  tests  to  measure  different  aspects  of  performance 
of  the  targeted  system.  Five  different  tests  are  executed  on  the  setups  specified  in  Figure  4.2: 
TCP_STREAM,  TCP_CRR,  TCP_CC  and  TCP_RR,  UDP_STREAM.  These  five  tests  form 
the  suite  of  benchmark  tests.  [30]  provides  additional  descriptions  of  the  various  tests.  Each  of 
these  will  be  discussed  in  their  respective  sections  below. 

We  made  use  of  the  “OMNI”  test  in  netperf  to  run  the  various  tests.  Previous  netperf  versions 
included  different  source  code  to  perform  different  tests.  The  “OMNI”  test  in  netperf  is  a 
consolidation  of  all  available  test  functions  into  a  single  test.  Different  flags  allow  different 
options  to  be  selected  to  enable  the  execution  of  specific  test  functions.  There  are  two  types  of 
flags:  global  and  test-specific.  “Global”  flags  (see  Table  4.3)  provide  options  that  apply  to  all 
benchmark  tests.  Test-specific  flags  are  described  in  their  respective  subsection. 

Table  4.3:  An  explanation  of  the  global  options  used  in  netperf 


Flags 

Value 

Description 

-H 

hostname 

This  option  sets  the  hostname  or  IP  address  of  the  targeted  sys¬ 
tem,  where  netserver  daemon  process  is  deployed.  This  option 
allows  us  to  measure  the  performance  of  the  two  different  physi¬ 
cal  servers  under  the  scenarios:  “F”  and  “F-”. 

-P 

0 

A  value  of  “0”  for  this  flag  disables  the  display  of  the  test  banner. 
The  test  banner  was  disabled  in  all  of  our  experiments  because  we 
executed  the  benchmark  tests  multiple  times  in  succession  and  the 
test  banners  are  redundant  and  unnecessarily  cluttered  the  output 

-P 

port 

This  option  sets  the  port  number  for  netperf  client  to  use  to  con¬ 
nect  to  the  netserver  daemon  process. 

-0 

filename 

This  option  allows  the  netperf  client  to  read  in  a  file  containing 
comma-separated  header  values.  These  headers  correspond  to  the 
different  output  values  we  want  to  record  in  our  results.  Exam¬ 
ples  of  these  headers  include  “THROUGHPUT”,  “THROUGH- 
PUTJJNITS”,  “PROTOCOL”  and  “ELAPSED_TIME”. 

4.4.1  TCP_STREAM 

MYSEA  clients  access  various  services  (i.e.,  web  and  mail)  hosted  on  the  MYSEA  server  us¬ 
ing  Transmission  Control  Protocol  (TCP)-based  protocols.  The  performance  of  MYSEA  in 
handling  TCP  packets  is  important  to  the  experience  of  the  client,  in  terms  of  the  latency  and 


27 


throughput  observed  by  the  client.  The  TCP_STREAM  test  is  designed  to  measure  the  per¬ 
formance  of  the  targeted  component  as  it  processes  TCP  data  packets.  The  test  provides  the 
throughput  in  bits  per  second  and  accounts  for  both  the  time  required  to  push  data  across  the 
network  and  the  time  to  process  it  on  the  targeted  component.  This  test  is  unidirectional  and  the 
time  spent  establishing  the  connection  is  not  included  in  the  throughput  calculation.  Table  4.4 
shows  the  test-specific  options  used  in  netperf  for  the  test:  TCP_STREAM. 

Table  4.4:  TCP_STREAM  test-specific  options 


Flags 

Value 

Description 

-d 

send 

This  option  sets  the  direction  of  the  test  relative  to  the  netperf 
process.  A  value  of  “send”  causes  netperf  to  send  data  to  the 
netserver  daemon  process,  resulting  in  a  unidirectional  test. 

-T 

TCP 

This  option  sets  “TCP”  to  be  the  protocol  used  for  the  test. 

4.4.2  TCP_CRR 

The  TCP_CRR  test  combines  TCP_CC  and  TCP_RR  and  measures  the  performance  of  the  tar¬ 
geted  component  in  handling  TCP  connections.  This  test  allows  us  to  measure  the  performance 
overhead  in  transactions  per  second  of  the  targeted  component  in  completing  the  entire  TCP  pro¬ 
cess:  establishing  a  connection  via  the  three-way  handshake,  transferring  one  byte  of  request, 
receiving  one  byte  of  response  and  tearing-down  the  connection.  We  can  then  obtain  the  latency 
of  one  complete  transaction  in  seconds  by  inverting  the  results.  This  test  is  designed  to  measure 
the  end-to-end  latency  experienced  by  a  client  when  making  a  service  request.  Table  4.5  shows 
the  test-specific  options  used  in  netperf  for  the  test:  TCP_CRR. 

Table  4.5:  TCP_CRR  test-specific  options 


Flags 

Value 

Description 

-c 

This  option  explicitly  instructs  netperf  to  include  connection  es¬ 
tablishment  and  teardown  in  its  measurement. 

-d 

send  1  recv 

This  option  sets  the  direction  of  the  test  relative  to  the  netperf  pro¬ 
cess.  A  value  of  “send”  causes  netperf  to  send  data  to  netserver 
daemon  process,  while  the  value  of  “recv”  causes  netserver  dae¬ 
mon  process  to  send  data  back  to  netperf.  This  results  in  a  bi¬ 
directional  receive  and  response  test. 

-T 

TCP 

This  option  sets  “TCP”  to  be  the  protocol  used  for  the  test. 
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4.4.3  TCP_CC 

The  TCP_CC  test  isolates  the  speed  at  which  connections  can  be  opened  and  closed  between 
the  target  and  client  using  the  TCP  protocol.  A  three-way  handshake  establishes  a  reliable 
connection  between  the  sender  and  receiver.  The  test  is  performed  in  a  synchronous  manner, 
with  no  data  transferred  between  the  two  entities.  This  test  continuously  opens  and  closes 
connections  between  netperf  and  netserver,  tracking  the  number  of  connections  created.  This 
test  provides  the  latency  in  transactions  per  second,  a  measurement  of  the  performance  of  the 
targeted  component  in  establishing  and  tearing-down  TCP  connections. 

The  MYSEA  trusted  proxy  introduces  additional  overhead  for  every  new  client  connection.  For 
each  new  client  connection,  a  proxy  SSS-C  process  is  spawned  to  handle  the  request.  This  test 
measures  the  impact  of  spawning  the  additional  process  to  handle  the  client  request.  Table  4.6 
shows  the  test-specific  options  used  in  netperf  for  the  test:  TCP_CC.  This  test  measures  the 
latency  associated  with  the  opening  and  closing  of  a  transaction. 


Table  4.6:  TCP  CC  test-specific  options 


Flags 

Value 

Description 

-c 

This  option  explicitly  instructs  netperf  to  include  connection  es¬ 
tablishment  and  teardown  in  its  measurement. 

-T 

TCP 

This  option  sets  “TCP”  to  be  the  protocol  used  for  the  test. 

4.4.4  TCP_RR 

The  TCP_RR  test  is  a  request  and  response  test  using  the  TCP  protocol,  where  each  request  and 
response  packet  is  one  byte  in  size.  It  is  bi-directional  and  measures  the  number  of  complete 
transactions  per  unit  time  that  can  be  exchanged  between  two  entities.  This  test  is  indepen¬ 
dent  of  the  time  taken  to  establish  the  initial  TCP  socket  connection  and  measures  the  latency 
associated  with  the  transfer  of  a  byte  of  request  and  response  request  between  the  client  and 
server.  Table  4.7  shows  the  test-specific  options  used  in  netperf  for  the  test:  TCP_RR.  This  test 
is  designed  to  measure  the  speed  for  uploading/downloading  large  files  to  MYSEA  server  (e.g., 
the  ftp  server),  where  the  initial  cost  of  setting  up  a  connection  is  less  significant. 

4.4.5  UDP_STREAM 

UDP  is  used  as  the  underlying  transport  protocol  for  VoIP,  and  some  other  services  provided 
by  MYSEA.  The  UDP_STREAM  test  is  similar  to  the  TCP_STREAM  test  except  that  UDP 
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Table  4.7:  TCP  RR  test-specific  options 


Flags 

Value 

Description 

-d 

send  1  recv 

This  option  sets  the  direction  of  the  test  relative  to  the  netperf 
process.  A  value  of  “send”  causes  netperf  to  send  data  to  net- 
server  daemon  process,  whereas  the  value  of  “recv”  causes  net- 
server  daemon  process  to  send  data  back  to  netperf.  Used  in  com¬ 
bination,  this  results  in  a  bi-directional  receive  and  response  test. 

-T 

TCP 

This  option  sets  “TCP”  to  be  the  protocol  used  for  the  test. 

is  used  as  the  mode  of  transport  rather  than  TCP.  This  test  provides  the  throughput  in  bits  per 
second  and  measures  the  performance  of  the  targeted  components  as  they  process  UDP  data 
packets.  Table  4.8  shows  the  test-specific  options  used  in  netperf  for  the  test:  UDPJSTREAM. 

Table  4.8:  UDP  STREAM  test-specific  options 


Flags 

Value 

Description 

-d 

send 

This  option  sets  the  direction  of  the  test  relative  to  the  netperf 
process.  A  value  of  “send”  causes  netperf  to  send  data  to  netserver 
daemon  process,  resulting  in  a  unidirectional  test. 

-T 

UDP 

This  option  sets  “UDP”  to  be  the  protocol  used  for  the  test. 

4.5  Test  Plan 

Netperf  client  runs  the  suite  of  benchmark  tests  on  the  following  six  setups:  “I  M-F”,  “I  M- 
F-”,  “I-M-F”,  “I-M-F-”,  “I-M  F”  and  “I-M  F-”  (See  Figure  4.2).  Each  benchmark  test  was 
executed  sequentially  for  ten  seconds  (netperf  default),  and  each  test  is  repeated  for  one  hundred 
trial  tests.  We  conducted  experiments  in  the  initial  laboratory  setup  and  repeated  the  trials  with 
different  numbers  of  times:  10,  50,  100,  1000,  10000.  One  hundred  (100)  trials  was  selected 
as  it  optimizes  the  amount  of  time  required  for  the  completion  of  the  entire  suite  of  benchmark 
tests  while  achieving  significant  population  samples  with  low  variances.  Before  any  execution 
of  benchmark  tests,  the  test  environment  was  restarted  and  prepared  for  the  respective  setup. 
Steps  included:  recompilation  of  the  netserver  source  code  for  the  respective  scenario  (“I-M  F”, 
“I-M  F-”,  “I-M  F”,  “I-M  F-”,  “I  M-F”  and  “I  M-F-”),  configuration  of  IPSec  to  setup/teardown 
the  secure  tunnel,  and  deployment  of  netserver  on  the  appropriate  server.  The  results  of  the  tests 
were  collected  at  the  end  of  experiment. 


30 


For  the  TCP_CC,  TCP_CRR  and  TCP_RR  benchmark  tests,  a  two-minute  delay  wait  period 
between  successive  tests  was  introduced  to  address  the  following  problem.  During  initial  trial 
experiments,  abnormal  results  were  obtained  when  these  tests  were  executed.  The  latency  of 
certain  trials  was  several  times  lower  than  the  rest  of  the  trials.  After  several  experiments,  it 
was  determined  to  be  an  issue  associated  with  a  TIME_WAIT  reuse  in  the  establishment  of  a 
socket  connection.  The  latency  drops  dramatically  when  the  test  reuses  a  connection  that  is 
in  the  TIME_WAIT  state.  This  results  in  a  non-trivial  delay  in  connection  establishment.  The 
two-minute  delay  wait  period  forced  the  TIME_WAIT  to  expire. 
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CHAPTER  5: 
Testing  and  Analysis 


This  chapter  discusses  the  execution  of  the  benchmark  tests  and  provides  analysis  of  the  results. 
Section  5.1  discusses  issues  related  to  the  collection  of  data  and  Section  5.2  provides  a  summary 
of  the  results. 


5.1  Overview 

Chapter  3  and  Chapter  4  discuss  the  methodology  for  designing  a  meaningful  benchmark  to 
measure  the  performance  of  targeted  components  in  MYSEA.  The  goal  is  to  achieve  soundness : 
developing  confidence  that  the  results  we  derive  from  our  measurements  are  well  justified,  with 
a  solid  understanding  of  the  strengths  and  limitations  of  the  measurement  process  on  which  we 
base  our  results  [31].  We  need  to  calibrate  the  test  results  by  examining  outliers  and  testing  for 
consistencies.  This  allows  us  to  reliably  reproduce  analysis  results  by  imposing  a  systematic 
structure  on  the  analysis  process. 

The  “OMNI”  test  in  netperf  allows  the  specification  of  the  type  of  output  values  to  collect  for 
the  test.  These  “selectors”  determine  the  data  to  be  reported.  Table  5.1  highlights  the  important 
selectors  used.  [30]  provides  the  detailed  documentation  for  the  “selectors”. 


Table  5.1:  Netperf  "OMNI”  test  selectors  used  during  benchmarking  MYSEA 


Selector  Name 

Description 

Examples 

PROTOCOL 

Protocol  used  in  the  bench¬ 
marks. 

UDP,  TCP 

DIRECTION 

Direction  of  the  data  flows 

relative  to  the  netperf  process. 

send  1  recv:  bidirectional  re¬ 
quest  and  response 
send:  unidirectional  request 

THROUGHPUT 

Throughput  for  the  test. 

350.0234 

THROUGHTPUT_UNITS 

Units  for  the  throughput 

10 3bits/s  or  10 6bits/s 
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5.2  Summary  of  Results 

We  compare  each  test  scenario,  relative  to  the  scenario  in  which  netserver  is  deployed  on  STOP 
without  the  network  being  protected  with  IPSec,  treating  this  as  a  “base  case”.  As  expected, 
we  see  that  the  base  case  shows  the  best  performance,  i.e.,  highest  throughput  and  lowest  la¬ 
tency  (see  Figure  5.1,  5.2  and  5.3).  For  TCP  stream  performance,  IPSec  encryption  results  in 
a  60%  decrease  in  throughput,  whereas  MYSEA  processes  result  in  between  21-24%  decrease 
in  throughput.  For  socket  related  performance,  IPSec  encryption  results  in  between  24-30% 
increase  in  latency,  whereas  MYSEA  processes  result  in  between  93-143%  increase  in  latency. 
These  observations  will  be  described  more  in  their  respective  sections. 

Our  UDP  results  are  inconclusive.  For  setups  without  IPSec  encryption,  the  throughput  values 
obtained  are  lower  than  when  IPSec  encryption  tunnel  is  used.  This  is  counter-intuitive  and 
possibly  the  result  of  packet  loss  during  testing.  We  refer  the  interested  reader  to  Appendix  G 
for  the  status  of  our  UDP  test  results.  We  leave  further  UDP  testing  as  future  work. 

5.2.1  Overhead  associated  with  IPSec 

IPSec  encryption  reduces  unidirectional  TCP  transfer  throughput  significantly.  The  perfor¬ 
mance  consequences  of  IPSec  in  an  architecture  similar  to  MYSEA,  with  server  gateway  and 
IPSec  encryption  of  traffic  between  TPE  and  server,  is  a  decrease  in  unidirectional  transfer 
speeds,  by  about  60%  (see  Table  5.3).  TCP  connect  latency  increases  an  average  of  24-30%, 
reflected  in  the  TCP_CC,  TCP_RR  and  TCP_CRR  tests.  For  the  details  of  each  test,  see  Ap¬ 
pendix  D. 

The  performance  of  IPSec  has  been  well  studied.  Shue  et  al  [32]  report  performance  overhead 
of  32-60%  for  Internet  Key  Exchange  (IKE)  based  IPSec  (Racoon  uses  IKE  based  IPSec)  and 
provides  a  comprehensive  study  on  the  performance  overhead  of  IPSec.  This  is  similar  to  our 
results  of  60%  decrease  in  TCP  throughput  and  socket  related  performance  overhead  range  of 
24-30%. 


5.2.2  Overhead  associated  with  the  MYSEA  Trusted  Proxy 

We  observe  a  21-25%  decrease  in  throughput  for  unidirectional  TCP  traffic,  relative  to  the  base 
case  (see  Table  5.4),  when  the  trusted  MYSEA  proxy  is  used  for  network  traffic  (for  details,  see 
Appendix  E).  We  believe  this  decrease  can  be  attributed  to  MYSEA’s  proxy  system  call  design, 
in  which  the  client  and  APS  communicate  via  a  proxy  process  (SSS-C)  mediating  all  accesses, 
inspecting  and  routing  TCP  network  packets  internally  before  reaching  the  APS. 
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Table  5.2:  Summary  of  the  average  overhead  witnessed  during  TCP  benchmarks  (Figure  5.1,  5.2, 
5.2) 


TCP_STREAM  Throughput 

Scenario 

Mean  (10b  bit  s/s) 

%  decrease 

MYSEA 

I-MF 

I-M  F- 

274.95 

270.57 

21.67% 

24.67% 

Base 

I-M-F 

I-M-F- 

351.00 

359.16 

-% 

-% 

IPSec 

I  M-F 

I  M-F- 

139.70 

144.51 

60.20% 

59.76% 

TCP_CC:  Latency  (ms/Trans) 

Scenario 

Mean  {Trans /s) 

Latency 

%  increase 

MYSEA 

I-MF 

I-M  F- 

529.54 

674.70 

1.888 

1.482 

143.30% 

93.22% 

Base 

I-MF 

I-M  F- 

1288.58 

1304.01 

0.776 

0.767 

-% 

-% 

IPSec 

I-MF 

I-M  F- 

993.89 

1020.16 

1.006 

0.980 

29.64% 

27.77% 

TCP_RR:  Latency  (ms/Trans) 

Scenario 

Mean  {Trans /s) 

Latency 

%  increase 

MYSEA 

I-MF 

I-M  F- 

2318.23 

2338.82 

0.431 

0.428 

20.40% 

19.55% 

Base 

I-MF 

I-M  F- 

2791.03 

2792.37 

0.358 

0.358 

-% 

-% 

IPSec 

I-MF 

I-M  F- 

2148.25 

2243.02 

0.465 

0.446 

29.89% 

24.58% 

The  latency  observed  when  the  establishment  of  a  new  connection  is  involved  is  anywhere 
between  90-150%,  relative  to  the  base  case  (see  Table  5.4),  whereas  TCP_RR  results  in  a 
smaller  increase  in  latency  at  about  20%,  similar  to  the  TCP__STREAM  performance  overhead. 
TCP_RR  is  similar  to  TCP_STREAM,  with  the  former  being  a  bidirectional  test.  The  TCP_RR 
and  TCP_STREAM  results  further  conclude  that  MYSEA  processes  increase  TCP  performance 
overhead  by  about  20%  as  a  consequence  of  its  proxy  design  to  route  all  network  packets  via  its 
respective  SSS-C  process. 

However,  performance  overhead  increases  significantly  to  90-150%  when  the  establishment  of 
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Table  5.3:  Summary  of  IPSec  results 


TCP_STREAM  Throughput 

Scenario 

Mean  (10 bbits/s) 

%  decrease 

F- 

I-M-F- 

I M-F- 

359.16 

144.51 

60.00% 

F 

I-M-F 

I  M-F 

351.00 

139.70 

60.20% 

TCP_CC:  Latency  (ms/Trans) 

Scenario 

Mean  (Trans /s) 

Latency 

%  increase 

F- 

I-M-F- 

I  M-F- 

1304.01 

1020.16 

0.767 

0.980 

27.77% 

F 

I-M-F 

I  M-F 

1288.58 

993.89 

0.776 

1.006 

29.64% 

TCP_RR:  Latency  (ms/Trans) 

Scenario 

Mean  (Trans /s) 

Latency 

%  increase 

F- 

I-M-F- 

I  M-F- 

2792.37 

2243.02 

0.358 

0.446 

24.58% 

F 

I-M-F 

I  M-F 

2791.03 

2148.25 

0.358 

0.465 

29.89% 

TCP_CRR:  Latency  (ms/Trans) 

Scenario 

Mean  (Trans /s) 

Latency 

%  increase 

F- 

I-M-F- 

I  M-F- 

1183.12 

903.96 

0.845 

1.106 

30.89% 

F 

I-M-F 

I  M-F 

1152.27 

889.10 

0.868 

1.125 

29.61% 

a  new  connection  is  involved.  We  believe  this  can  be  attributed  to  the  creation  of  a  new  SSS-C 
process  for  every  new  TCP  connection,  resulting  in  significant  performance  overhead. 

5.2.3  MYSEA  Federation  Overhead 

When  a  MYSEA  service  (APS)  and  the  TPS  process  are  deployed  on  different  servers,  the 
APS’s  proxy  causes  a  Federated  request  to  fetch  information  about  the  client’s  current  level  from 
the  MYSEA  server  hosting  the  TPS.  In  practice,  these  security  checks  occur  per  connection  (for 
UDP,  per  packet).  In  our  testing,  the  netserver  process  acts  as  an  APS,  allowing  us  to  interpret 
the  overhead  associated  with  the  intra-Federation  communication. 


36 


Table  5.4:  Summary  of  MYSEA  results 


TCP_STREAM  Throughput 

Scenario 

Mean  (10P  bit  s/s) 

%  decrease 

F- 

I-M-F- 

I-M  F- 

359.16 

270.57 

24.67% 

F 

I-M-F 

I M-F 

351.00 

274.95 

21.67% 

TCP_CC:  Latency  (ms/Trans) 

Scenario 

Mean  (Trans /s) 

Latency 

%  increase 

F- 

I-M-F- 

I  M-F- 

1304.01 

674.70 

0.767 

1.482 

93.22% 

F 

I-M-F 

I  M-F 

1288.58 

529.54 

0.776 

1.888 

143.30% 

TCP_RR:  Latency  (ms/Trans) 

Scenario 

Mean  (Trans /s) 

Latency 

%  increase 

F- 

I-M-F- 

I  M-F- 

2792.37 

2338.82 

0.358 

0.428 

19.55% 

F 

I-M-F 

I  M-F 

2791.03 

2318.23 

0.358 

0.431 

20.39% 

TCP_CRR:  Latency  (ms/Trans) 

Scenario 

Mean  (Trans /s) 

Latency 

%  increase 

F- 

I-M-F- 

I  M-F- 

1183.12 

575.88 

0.845 

1.736 

105.44% 

F 

I-M-F 

I  M-F 

1152.27 

467.30 

0.868 

2.140 

146.54% 

For  TCP  services,  we  observe  a  greater  latency  when  the  APS  and  TPS  are  deployed  on  different 
servers.  From  the  results  of  the  TCP_CC  and  TCP_CRR  tests,  setup  “F”  shows  a  significant 
increase  in  latency  of  about  50%  compared  to  setup  “F-”.  We  believe  this  can  be  attributed 
to  the  additional  performance  overhead  associated  with  the  MYSEA  processes  communicating 
internally  when  the  APS  and  TPS  are  on  different  servers. 
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Figure  5.1:  Boxplot  showing  the  summary  of  TCP_STREAM  performance  for  the  six  setups 
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Figure  5.2:  Boxplot  showing  the  summary  of  TCP_CC  socket  performance  for  the  six  setups 
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Figure  5.3:  Boxplot  showing  the  summary  of  TCP  RR  request  and  response  performance  for  the  six 
setups 
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CHAPTER  6: 
Conclusion 


We  analyzed  the  architecture  of  the  MYSEA  framework,  and  identified  the  major  components 
and  services  that  create  a  high  assurance  MLS  environment.  Overall,  we  observe  that  the  MY¬ 
SEA  trusted  proxy  architecture  incurs  significant  overhead  (impacting  throughput  and  latency 
for  network  connections).  On  average,  the  latency  for  new  connections  more  than  doubles 
(about  118%)  compared  to  MYSEA’s  underlying  platform.  These  costs  are  associated  with 
connection  setup,  i.e.  starting  a  socket  proxy  on  behalf  of  the  user.  Once  a  connection  is  es¬ 
tablished,  we  observe  an  average  throughput  decrease  of  about  23%  for  unidirectional  TCP 
streams,  associated  with  the  cost  of  proxying  a  socket  write()  request.  In  a  Federated  setting, 
the  way  state  is  communicated  among  servers  alone  is  associated  with  an  increase  in  latency  of 
about  50%. 

6.1  Recommendations  for  Future  Work 

This  thesis  performed  a  high  level  analysis  on  the  performance  of  MYSEA,  focusing  on  the 
comparison  of  the  mean  latency  values  and  throughput.  Further  detailed  mathematical  analysis 
can  be  performed  on  the  data,  finding  other  trends  for  the  current  set  of  data.  The  UDP  results 
are  not  conclusive,  and  future  work  can  look  into  the  possible  causes  and  mitigations  to  study 
the  performance  overhead  of  UDP  packets.  Future  work  can  analyze  other  components  and 
services  in  MYSEA,  such  as  the  performance  of  the  TPE  and  server  gateway.  Future  work  can 
explore  recommendations  for  improving  the  performance  of  the  bottlenecks  identified.  Future 
work  can  also  measure  the  performance  of  MYSEA  in  setups  “I  M  F-”  and  “I  M  F”,  which  are 
closer  comparisons  to  the  operational  environment  that  MYSEA  is  deployed  in. 
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APPENDIX  A: 
Netserver  Modifications 


This  appendix  describes  the  modifications  to  netserver  to  operate  in  the  MYSEA  environment. 

1.  Include  MYSEA  headers 

In  the  header  file:  netlib.h,  insert  the  following  include  statement. 

#include  "mysea_portability.h" 

2.  Insert  flags  for  different  compilation  modes 

Flags  are  used  for  the  compilation  of  netserver  in  its  two  different  modes  of  operation: 
inetd  and  daemon,  inetd  mode  is  used  in  the  MYSEA  environment,  under  scenario  “M", 
while  daemon  mode  is  used  in  STOP,  under  scenario  “M-".  We  need  to  compile  the 
netserver  source  code  with  support  for  MYSEA  processes  in  scenario  “M".  Any  modifi¬ 
cations  made  to  the  source  code  to  operate  in  MYSEA  environment  use  flags  inserted  on 
the  command  line  to  obtain  the  appropriate  compilation. 


#if def  USE_M_SQCKET 

#include  "niysea_portability .  h" 

#endif 


Figure  A.l:  Insertion  of  MYSEA  header  with  flags  to  compile  for  MYSEA  support 

3.  Insert  necessary  code 

Additional  code  and  functions  are  added  to  netserver  to  integrate  with  MYSEA  processes. 
The  following  functions  and  headers  are  added  to  the  source  code: 


#if def  USE_M_SOCKET 
int  en y_f cf  =  -1; 
#endif 


void  exit_mysea(void) 

{ 

#if def  USE_M_SOCKET 
mport_exit( 0) : 
#endif  //  USE_M_50CKET 
exiti 0) ; 

> 


Figure  A. 2:  Addition  of  headers  to  source  code  in  file:  netserver. c 
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void 

set  mysea  server  sock(int  mskt  fd)  { 

#if def  WIN32 

server  sock  =  ( SOCKET) GetStdHandlef STD 
#elif  ! defined(  VMS) 

if  fserver_sock  !=  INVALID_SOCKET)  { 
printfC'yo,  Iz  ain't  invalid ! \n" ) ; 
exit(l) ; 

INPUT_ HANDLE)  : 

.r 

server_sock  =  fliskt_fd; 

#else 

printf f "\tALL  ELSEVn" ) ; 

if  t  (server_sock  =  fnskt_Tcf )  ==  INVALID_SQCKET)  { 
f printf f  stder r r 

"%s:  failed  to  grab  aux  server  socket:  Sfcs  (errno  %s)\nMp 
_ FUNCTION _ _p 

st rerrorf errno) p 
errno) ; 
f f lushC stderr ) ; 
exit (1) ; 
l 

#endif 

> 

Figure  A. 3:  Addition  of  functions  to  source  code  in  file:  netserver.c,  function:  main 
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int  _cdecl| 

main( int  argcP  char  *argv[])  { 

#ifdef  USE_H_SOCKET 
int  mysea_exit  =  0; 
int  result: 

//  Initialize  underlying  MYSEA  modules 
result  =  mport_init ( APS_M0DEr  "netserver_log" ,  NULL): 
if  ( result  !=  MPORTJDK)  { 
result  =  -1; 

printf ("mport_init  error\n"); 
mysea_exit  =  1; 

}else{  //  find  the  MSKT  fd 

result  =  mport_get_f d(&my_fd) : 
if  (result  1  =  HP0RT_0K)-[ 
my_fd  =  -1: 

printf (Mmport_get_f d  error\n") ; 
mysea_exit  -  1; 

}else{ 

result  =  mpo rt_get_f ile_st ream (Sphere) : 
if  (  result  !  =  HPORT_0K)-[ 

printf [Mmport_get_file_st ream  error") ; 
mysea_exit  =  1; 

}else{ 

f printf  (vjhereP  "Using  MSKT  fd  [%d]\n"p  my_fd)  : 

> 

> 

> 

if  [mysea_exit ) { 

printf ("Init izalition  error,  exiting"): 
mport_exit ( 1) : 

> 

atexit (&exit_mysea) : 

/^result  stores  the  mskt_fd  */ 

#endif  //  USE_M_SOCKET 
#if def  WIN32 


Figure  A. 4:  Modifications  to  source  code  in  file:  netserver.c,  function:  main 
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#ifdef  USE_H_SOCKET 

set_mysea_server_sock(my_f d)  ; 

/^open_debug_f ile( )  ;+/ 

process_requests( ) ; 

#else 

if  (child)  { 

/+  we  are  the  child  of  either  an  inetd  or  parent  netserver  via 
spawning  (Windows)  rather  than  forkding.  if  we  were  forkOed 
we  would  not  be  coming  through  this  way.  set_server_sock( )  must 
be  called  before  open_debug_f ile( )  or  there  is  a  chance  that 
we'll  toast  the  descriptor  when  we  do  not  wish  it.  */ 
set_server_sock( ) ; 
open_debug_f ilef ) : 
process_requests( )  ; 

l 

else  if  (daemon_parent )  { 

/*  we  are  the  parent  daemonized  netserver 
process.  accept_connections[ )  will  decide  if  we  want  to  spawn  a 
child  process 
accept_connections( ) ; 

1 

else  | 

/ *  we  are  the  top  netserver  process,  so  we  have  to  create  the 
listen  endpoint(s)  and  decide  if  we  want  to  daemonize  */ 
setup_listens( local_hos t_name, listen_port , local_add  ress_f amily ) ; 
if  (want_daemonize)  { 
daemonize ( ) ; 

> 

accept_connections( ) ; 

>i 

#endif 


Figure  A. 5:  Modifications  to  source  code  in  file:  netserver. c,  function:  main 
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APPENDIX  B: 

Installing  and  Running  the  Benchmark  Tool 


Two  files  are  necessary  for  the  installation  and  execution  of  Netperf  benchmark  tool:  stop.vl.O.tar 
and  linux.vl.O.tar. 

stop.vl.O.tar  is  used  for  the  installation  of  netserver  process  on  STOP  server,  while  linux.vl.O.tar 
is  used  for  the  installation  of  netperf  client  and  execution  of  benchmark  tests  on  the  TPE.  Both 
are  under  configuration  management  within  the  MYSEA  project. 

B.l  Installing  Netserver 

1.  Decompress  stop.vl.O.tar 


tar  -xf  stop.vl.O.tar 


2.  Select  mode  of  installation 

Inetd  is  used  for  the  setup  “M"  while  daemon  is  used  for  the  setup  “M-".  The  default 
mode  is  inetd  mode. 


inetd  mode: 

. /mysea_install_netserver . sh  -m  inetd 

daemon  mode: 

./mysea_install_netserver.sh  -m  daemon 

3.  Optional 

Specify  the  control  port  number  for  netserver  to  monitor  on  the  network.  The  default  is 
12865. 


./mysea_install_netserver.sh  -cp  10000 


Specify  the  help  flag  “h"  for  more  options. 


. /mysea_install_netserver . sh  -h 
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B.2  Installing  Netperf 

1.  Decompress  linux.vl.O.tar 


tar  -xf  linux.vl.O.tar 


2.  Optional 

Specify  the  help  flag  “h"  for  more  options. 

. /mysea_install_netperf . sh  -h 


B.3  Execute  benchmark  tests 

1.  Installing  Netserver  and  Netperf 

Refer  to  Section  B.l  and  Section  B.2  for  the  installation  details 

2.  Execute  benchmark  tests 

Provide  the  necessary  flags  for  the  execution  of  different  tests. 


. /mysea_run_netperf . sh 

Flags 

-cp | conPort 

Control  port  that  netserver  is  monitoring 

-dp | dataPort 

Data  port  used  for  transmission  of  data  between  Netserver 

and  Netperf 

-H  Ihost 

IP  address  or  hostname  of  the  targeted  machine,  where 

netserver  server  process  is  residing 

-n  | numTests 

Number  of  times  each  test  is  executed 

-t  Itest 

Name  of  the  test  to  be  executed 

-tl | testlen 

The  length  of  time  in  seconds  each  test  is  executed. 

3.  Collect  Results 

Netperf  collects  the  results  from  the  tests  and  places  them  in  the  current  directory.  The 
default  format  of  the  output  file  is: 

result- [test] - [testlen] - [numTests] -yyyymmddHHMMSS 
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4.  Optional 

Specify  the  help  flag  “h"  for  more  options. 

. /mysea_run_netperf . sh  -h 
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APPENDIX  C: 
Overview  of  Results 


This  appendix  contains  the  boxplot  summaries  of  all  the  testing.  The  boxplots  compare  the 
results  of  the  benchmark  tests  executed  under  all  the  scenarios. 
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Figure  C.l:  Boxplots  of  TCP_STREAM  Tests  of  all  scenarios 
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Figure  C.3:  Boxplots  of  TCP_CC  Tests  of  all  scenarios 
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Figure  C.4:  Boxplots  of  TCP_CRR  Tests  of  all  scenarios 
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Figure  C.5:  Boxplots  of  TCP_RR  Tests  of  all  scenarios 
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APPENDIX  D: 

Results  of  IPSec  Comparison 


This  appendix  contains  the  results  of  the  benchmark  tests  used  for  the  analysis  of  the  perfor¬ 
mance  overhead  of  IPSec. 
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Figure  D.l:  Boxplots  comparison  of  IPSec  using  TCP_STREAM  Tests 
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Figure  D.2:  Histogram  comparison  of  IPSec  using  TCP_STREAM  Tests 
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Figure  D.3:  Scatterplot  comparison  of  IPSec  using  TCP_STREAM  Tests 
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Figure  D.4:  Boxplots  comparison  of  the  performance  of  IPSec  using  UDP_STREAM  Tests 
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Figure  D.5:  Histogram  comparison  of  the  performance  of  IPSec  using  UDP_STREAM  Tests 
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Figure  D.6:  Scatterplot  comparison  of  the  performance  of  IPSec  using  UDP_STREAM  Tests 
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Figure  D.7:  Boxplots  comparison  of  the  performance  of  IPSec  using  TCP_CC  Tests 
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Figure  D.8:  Histogram  comparison  of  the  performance  of  IPSec  using  TCP_CC  Tests 
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Figure  D.9:  Scatterplot  comparison  of  the  performance  of  IPSec  using  TCP_CC  Tests 
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Figure  D.10:  Boxplots  comparison  of  the  performance  of  IPSec  using  TCP_CRR  Tests 
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Figure  D.  11:  Histogram  comparison  of  the  performance  of  IPSec  using  TCP  CRR  Tests 
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Figure  D.12:  Scatterplot  comparison  of  the  performance  of  IPSec  using  TCP  CRR  Tests 


69 


TCP  RR 


THROUGHPUT  vs  Scenarios 


Figure  D.13:  Boxplots  comparison  of  the  performance  of  IPSec  using  TCP  RR  Tests 
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Figure  D.14:  Histogram  comparison  of  the  performance  of  IPSec  using  TCP_RR  Tests 
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Figure  D.15:  Scatterplot  comparison  of  the  performance  of  IPSec  using  TCP_RR  Tests 
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Figure  D.16:  Boxplots  comparison  of  IPSec  using  TCP_STREAM  Tests 
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Figure  D.17:  Histogram  comparison  of  IPSec  using  TCP  STREAM  Tests 
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Figure  D.18:  Scatterplot  comparison  of  IPSec  using  TCP  STREAM  Tests 
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Figure  D.19:  Boxplots  comparison  of  the  performance  of  IPSec  using  UDP_STREAM  Tests 
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Figure  D.20:  Histogram  comparison  of  the  performance  of  IPSec  using  UDP  STREAM  Tests 
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Figure  D.21:  Scatterplot  comparison  of  the  performance  of  IPSec  using  UDP  STREAM  Tests 
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Figure  D.22:  Boxplots  comparison  of  the  performance  of  IPSec  using  TCP  CC  Tests 
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Figure  D.23:  Histogram  comparison  of  the  performance  of  IPSec  using  TCP  CC  Tests 
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Figure  D.24:  Scatterplot  comparison  of  the  performance  of  IPSec  using  TCP  CC  Tests 
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Figure  D.25:  Boxplots  comparison  of  the  performance  of  IPSec  using  TCP_CRR  Tests 
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Figure  D.26:  Histogram  comparison  of  the  performance  of  IPSec  using  TCP  CRR  Tests 
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Figure  D.27:  Scatterplot  comparison  of  the  performance  of  IPSec  using  TCP  CRR  Tests 
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Figure  D.28:  Boxplots  comparison  of  the  performance  of  IPSec  using  TCP_RR  Tests 
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Figure  D.29:  Histogram  comparison  of  the  performance  of  IPSec  using  TCP_RR  Tests 
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Figure  D.30:  Scatterplot  comparison  of  the  performance  of  IPSec  using  TCP_RR  Tests 


87 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


88 


APPENDIX  E: 

Results  of  MYSEA  Comparison 


This  appendix  contains  the  results  of  the  benchmark  tests  used  for  the  analysis  of  the  perfor¬ 
mance  overhead  of  the  MYSEA  proxy. 
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Figure  E.l:  Boxplots  comparison  of  MYSEA  processes  using  TCP  STREAM  Tests 
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Figure  E.2:  Histogram  comparison  of  MYSEA  processes  using  TCP_STREAM  Tests 
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Figure  E.3:  Scatterplot  comparison  of  MYSEA  processes  using  TCP_STREAM  Tests 


92 


400 

300 

200 

100 

0 

■100 

200 

300 

400 

■e  E. 


UDP  STREAM 


THROUGHPUT  vs  Scenarios 


153.59 

| 

223.^9 


+ 

+ 


+ 

4- 


268.79 


118.  t 


219  19 


+ 

t 


I 

+ 


25.6 


Mean: 


Mean: 


305,2  fr . . . 140,3.4. 


I-M-F- 


l-MF- 


Scenarios 


4:  Boxplots  comparison  of  the  performance  of  IPSec  using  UDP_STREAM  Tests 
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Figure  E.5:  Histogram  comparison  of  the  performance  of  IPSec  using  UDP  STREAM  Tests 
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Figure  E.6:  Scatterplot  comparison  of  the  performance  of  IPSec  using  UDP_STREAM  Tests 
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Figure  E.7:  Boxplots  comparison  of  the  performance  of  IPSec  using  TCP  CC  Tests 
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Figure  E.8:  Histogram  comparison  of  the  performance  of  IPSec  using  TCP_CC  Tests 


97 


TCP  CC 


Figure  E.9:  Scatterplot  comparison  of  the  performance  of  IPSec  using  TCP  CC  Tests 
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Figure  E.10:  Boxplots  comparison  of  the  performance  of  IPSec  using  TCP_CRR  Tests 
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Figure  E.ll:  Histogram  comparison  of  the  performance  of  IPSec  using  TCP  CRR  Tests 
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Figure  E.12:  Scatterplot  comparison  of  the  performance  of  IPSec  using  TCP  CRR  Tests 
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Figure  E.13:  Boxplots  comparison  of  the  performance  of  IPSec  using  TCP_RR  Tests 
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Figure  E.14:  Histogram  comparison  of  the  performance  of  IPSec  using  TCP  RR  Tests 
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Figure  E.15:  Scatterplot  comparison  of  the  performance  of  IPSec  using  TCP_RR  Tests 
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Figure  E.16:  Boxplots  comparison  of  MYSEA  processes  using  TCP_STREAM  Tests 
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Figure  E.17:  Histogram  comparison  of  MYSEA  processes  using  TCP  STREAM  Tests 
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Figure  E.18:  Scatterplot  comparison  of  MYSEA  processes  using  TCP_STREAM  Tests 
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Figure  E.19:  Boxplots  comparison  of  the  performance  of  IPSec  using  UDP_STREAM  Tests 
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Figure  E.20:  Histogram  comparison  of  the  performance  of  IPSec  using  UDP  STREAM  Tests 
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Figure  E.21:  Scatterplot  comparison  of  the  performance  of  IPSec  using  UDP  STREAM  Tests 
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Figure  E.22:  Boxplots  comparison  of  the  performance  of  IPSec  using  TCP  CC  Tests 
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Figure  E.23:  Histogram  comparison  of  the  performance  of  IPSec  using  TCP  CC  Tests 
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Figure  E.24:  Scatterplot  comparison  of  the  performance  of  IPSec  using  TCP  CC  Tests 
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Figure  E.25:  Boxplots  comparison  of  the  performance  of  IPSec  using  TCP_CRR  Tests 
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Figure  E.26:  Histogram  comparison  of  the  performance  of  IPSec  using  TCP  CRR  Tests 
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Figure  E.27:  Scatterplot  comparison  of  the  performance  of  IPSec  using  TCP  CRR  Tests 
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Figure  E.28:  Boxplots  comparison  of  the  performance  of  IPSec  using  TCP_RR  Tests 
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Figure  E.29:  Histogram  comparison  of  the  performance  of  IPSec  using  TCP_RR  Tests 
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Figure  E.30:  Scatterplot  comparison  of  the  performance  of  IPSec  using  TCP_RR  Tests 
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APPENDIX  F: 

Results  of  APS  and  TPS  on  different  servers 


This  appendix  contains  the  results  of  the  benchmark  tests  used  for  the  analysis  of  the  perfor¬ 
mance  overhead  of  the  MYSEA  Federation. 
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Figure  F.l:  Boxplots  comparison  of  APS  and  TPS  on  different  servers  using  TCP_STREAM  Tests 
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Figure  F.2:  Histograms  of  APS  and  TPS  on  different  servers  using  TCP_STREAM  Tests 
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Figure  F.3:  Scatterplots  of  APS  and  TPS  on  different  servers  using  TCP_STREAM  Tests 
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Figure  F.4:  Boxplots  comparison  of  APS  and  TPS  on  different  servers  using  UDP_STREAM  Tests 
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Figure  F.5:  Histograms  of  APS  and  TPS  on  different  servers  using  UDP_STREAM  Tests 
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Figure  F.6:  Scatterplots  comparison  of  APS  and  TPS  on  different  servers  using  UDP  STREAM  Tests 
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Figure  F.7:  Boxplots  comparison  of  APS  and  TPS  on  different  servers  using  TCP_CC  Tests 
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Figure  F.8:  Histograms  of  APS  and  TPS  on  different  servers  using  TCP  CC  Tests 
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Figure  F.9:  Scatterplots  comparison  of  APS  and  TPS  on  different  servers  using  TCP  CC  Tests 
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Figure  F.10:  Boxplots  comparison  of  APS  and  TPS  on  different  servers  using  TCP_CRR  Tests 
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Figure  F.ll:  Histograms  of  APS  and  TPS  on  different  servers  using  TCP_CRR  Tests 
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Figure  F.12:  Scatterplots  comparison  of  APS  and  TPS  on  different  servers  using  TCP_CRR  Tests 
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Figure  F.13:  Boxplots  comparison  of  APS  and  TPS  on  different  servers  using  TCP_RR  Tests 
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Figure  F.14:  Histograms  of  APS  and  TPS  on  different  servers  using  TCP  RR  Tests 
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Figure  F.15:  Scatterplots  comparison  of  APS  and  TPS  on  different  servers  using  TCP_RR  Tests 


APPENDIX  G: 
UDP  Discussion 


The  results  of  UDP_STREAM  test  show  abnormalities  in  its  throughput.  Figure  G.  1  shows  the 
boxplot  summary  of  test  results  for  UDP_STREAM  test.  The  throughput  mean  values  for  the 
scenarios  when  IPSec  encryption  tunnel  is  not  deployed  is  smaller  than  when  the  IPSec  encryp¬ 
tion  tunnel  is  deployed.  This  is  counter  intuitive  as  the  presence  of  IPSec  encryption  tunnel 
encrypting  UDP  packets  should  result  in  a  additional  overhead,  and  thus  reduce  its  throughput. 
Upon  analysis  of  the  raw  test  data  for  the  scenario  when  NO  IPSec  encryption  is  used,  we  dis¬ 
covered  that  the  throughput  is  low  because  of  the  large  disparity  between  the  number  of  udp 
packets  sent  by  netperf  and  the  number  of  udp  packets  received  by  netserver.  Table  G.l  shows 
the  number  of  udp  packets  sent  and  received  when  no  IPSec  encryption  is  used. 

In  addition,  Table  G.2  shows  the  test  results  for  the  scenario  when  IPSec  encryption  is  used.  We 
see  a  greater  than  50%  decrease  in  the  number  of  UDP  packets  received. 

Table  G.l:  Examples  of  UDP  STREAM  Test  raw  data  for  NO  IPSec  encryption 


No. 

local  send  throughput 
(10  3bits/s) 

remove  recv  throughput 
(10  3bits/s) 

local  packets  sent 

remote  packets  recv 

1 

960418.23 

998.37 

75035 

78 

2 

960815.57 

25.6 

75066 

2 

3 

960764.19 

153.6 

75062 

12 

4 

960760.61 

25.6 

75062 

2 

5 

6 

The  following  are  the  suggested  steps  for  future  work. 

1.  Use  a  network  analyzer  to  capture  the  UDP  packets  Verify  that  the  number  of  packets 
captured  on  both  sending  and  receiving  party  is  accurately  reported  by  netserver. 

2.  Analyze  the  function  in  source  code:  recv_omni  in  nettest_omni.c  An  error  message  was 
reported  in  this  function:  “  YO!  TIMESUP!”.  Verify  that  the  system  calls  that  netserver 
use  for  UDP  related  tests  are  valid  in  STOP  OS. 
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Table  G.2:  Examples  of  UDP_STREAM  Test  raw  data  when  IPSec  encryption  is  used 


No. 

local  send  throughput 
(XQpbits/s) 

remove  recv  throughput 
(103  bit  s/s) 

local  packets  sent 

remote  packets  recv 

1 

326865.51 

141674.15 

25538 

11069 

2 

327094.31 

155394.12 

25556 

12141 

3 

329015.93 

149571.31 

25706 

11686 

4 

327903.53 

149546.23 

25619 

11684 

5 

6 
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Figure  G.l:  Boxplot  showing  the  summary  of  test  results  for  UDP_STREAM  test 
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